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DETAILED ACTION 
Response to Arguments 

1 . The Request for Continued Examination filed 1 1/05/2007 have been entered and 
made of record. 

2. The Declaration under 37 C. F. R. 1 . 1 32 filed 1 0/09/2007 have been entered and 
made of record. Further discussions toward the declaration are addressed below. 

3. The Applicant has canceled claim(s) 4. 

4. The application has pending claim (s) 1-3 and 5-7. 

5. Applicant's arguments with respect to claim(s) 1-3 and 5-7 have been considered 
but are moot in view of the new ground(s) of rejection because of the Request for 
Continued Examination (RCE). 

6. Applicant's arguments filed 1 1/05/2007 have been fully considered but they are 
not persuasive. 

The Applicant alleges, "A notable feature of Claim 1 ..." in page 5, "The Office 
Action cites the Vivarelli article ..." in page 5, and "Accordingly, the neural networks ..." 
in page 5, and states respectively that firstly the prior art references Tretter and Vivarelli 
alone or in combination do not suggest the amended claim 1 which now incorporates a 
more specific version of the limitations of original claim 4, "a Bayesian network 
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configured to produce the final estimate of image class with evidence missincf . The 
Applicant secondly suggests that the accompanying Declaration under 37 C.F.R. 1.132 
by one of the inventor's, Jiebo Luo, shows that the neural network of Tretter and the 
Bayesian neural network of Vivarelli both require a complete input set for computation 
which goes against the amended claim limitation that the Bayesian network is 
configured to produce a result even with evidence missing. Firstly , the Examiner 
acknowledges that the Vivarelli reference does not specifically suggest that Vivarelli's 
Bayesian neural network is configured to produce a result with evidence missing, 
however the Examiner disagrees and does believe that Tretter suggests a neural 
network configured to produce the final estimate of image class with evidence missing 
and therefore the Vivarelli reference is only used as a secondary reference to further 
suggest that Tretter's neural network may be a Bayesian neural network. Tretter 
teaches that Tretter's neural network consists of several nodes and that each node is 
configured to receive for example one meta-data [focusing distance for example] (see 
Tretter, paragraph 0029, lines 7-10). Tretter's number of nodes depends on the desired 
accuracy for the classification scheme that the neural network produces (see Tretter, 
paragraph 0033, lines 1-6) and Tretter gives an example of having a neural network 
consisting of six decision-making nodes for a specific desired accuracy (see Tretter, 
paragraph 0033, lines 1-6) and Tretter also suggests that the meta-data may consist of 
but not limited to an automatic gain setting, film speed, shutter speed, white balance, 
aperture/lens index, focusing distance, date, time, and flash/no flash [at least 9 different 
meta-data's] (see Tretter, paragraph 0008, lines 9-12). Therefore if Tretter wanted a 
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specific desired accuracy which required 6 nodes [each node requires one different 
meta-data such as focusing distance for example as discussed above], there would 
definitely be at least 3 different meta-data tags which would not be addressed and 
therefore the neural network would be missing at least 3 different meta-data tag 
evidence. Secondly in regards to the Declaration under 37 C.F.R. 1 .132, the inventor 
Jiebo Luo starts by stating that the Vivarelli Bayesian neural network cannot perform its 
computations when any part of the data from the particular set of input information is 
missing [see Declaration, page 1, Point 6], but then clearlv states that a conventional 
Bavesian network is trained to be able to perform when data of input information is 
missing [see Declaration, page 1 , Point 7]. Therefore the inventor clearly admits that 
the amended limitation is conventional in that a Bayesian network may perform with 
evidence missing. Therefore claims 1-3, and 5-7 are still not in condition for allowance; 
the claim rejections are further discussed below. 

The Applicant alleges, "In anticipation of the Examiner substituting ..." in pages 
5-6, and states respectively that the prior art reference Tretter does not teach a 
Bayesian network which produces a result with evidence missing and therefore if the 
Examiner substitutes the Vivarelli article with a reference that does teach that claim 
limitation, the Examiner is using hindsight to modify the Tretter publication. Firstly , the 
Examiner disagrees as discussed above and does believe that Tretter suggests a 
neural network configured to produce the final estimate of image class with evidence 
missing and therefore the Vivarelli reference is only used as a secondary reference to 
further suggests that Tretter's neural network may be a Bayesian neural network. 
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Secondly as discussed above, the inventor clearly admits in the Declaration under 37 
C.F.R. 1.132 that a Bayesian network is trained to be able to perform when data of input 
information is missing which further supports the Examiner's obviousness rejection. 
Thirdly , in response to applicant's argument that the Examiner's conclusion of 
obviousness is based upon improper hindsight reasoning, it must be recognized that 
any judgment on obviousness is in a sense necessarily a reconstruction based upon 
hindsight reasoning. But so long as it takes into account only knowledge which was 
within the level of ordinary skill at the time the claimed invention was made, and does 
not include knowledge gleaned only from the applicant's disclosure, such a 
reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 
1971). Therefore claims 1-3, and 5-7 are still not in condition for allowance; the claim 
rejections are further discussed below. 



Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S. C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. Claims 1-3 and 5 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Tretter (US 2002/0140843 A1 , as applied in previous Office Action) in view of 
Vivarelli ("Using Bayesian neural networks to classify segmented images" - IEEE - July 
1997, pages 268-273, as applied in previous Office Action). 
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Re Claim 1: Tretter discloses a method for scene classification / content-based and 
metadata based classification (see abstract, lines 1-3) of a digital image comprising the 
steps of (a) extracting one or more pre-determined camera metadata tags / meta data 
from the digital image (see abstract, lines 6-10, paragraph [0008], lines 9-12, meta-data 
may consist of but not limited to an automatic gain setting, film speed, shutter speed, 
white balance, aperture/lens Index, focusing distance, date, time, and flash/no flash [at 
least 9 different meta-data's]); (b) obtaining an estimate of image class / meta data 
based classification based on (1 ) the extracted camera metadata tags / meta data (see 
abstract, lines 1-3, 10-14, paragraph [0008], lines 15-20) and not (2) image content 
features (paragraph [0008] shows that the meta-data classifier only uses data capturing 
attributes such as shutter speed, focusing distance, date and time, flash/no flash), 
thereby providing a metadata-based estimate / meta-data based classification; (c) 
obtaining, separately from the metadata-based estimate (paragraphs [0003]-[0004] and 
[0009] shows that the content based classification only uses content of the subject data 
such as color and contrast) another estimate of image class / content based 
classification of the digital image based on (1) image content features / content based 
data (see abstract, lines 1-3, 10-14, paragraph [0008], lines 15-20) and not (2) the 
extracted camera metadata tags (paragraphs [0003]-[0004] and [0009] show that the 
content based classification only uses content of the subject data such as color and 
contrast), thereby providing an image content-based estimate / content based 
classification; and (d) producing a final estimate of image class / combination of both 
results of the content based classifier and meta data based classifier of the digital image 
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based on a combination of the metadata-based estimate / meta data based 
classification and the image content-based estimate / content based classification (see 
abstract, lines 10-14, paragraph [0008], lines 15-20), wherein the combination in step 
(d) is obtained by using a network / nodal, neural network (see paragraph [0032], lines 6- 
11) configured to produce the final estimate of image class with evidence missing / 
missing meta-data evidence (see paragraph [0029], lines 7-10, paragraph [0008], lines 
9-12, Tretter's neural network consists of several nodes and each node is configured to 
receive for example one meta-data [focusing distance for example, other meta-data 
may consist of but not limited to an automatic gain setting, film speed, shutter speed, 
white balance, aperture/lens index, focusing distance, date, time, and flash/no flash {at 
least 9 different meta-data}], Tretter's number of nodes depends on the desired 
accuracy for the classification scheme that the neural network produces {see 
paragraph [0033], lines 1-6} and Tretter gives an example of having a neural network 
consisting of six decision-making nodes for a specific desired accuracy {see paragraph 
[0033], lines 1-6}, therefore if Tretter wanted a specific desired accuracy which required 
6 nodes [each node requires one different meta-data such as focusing distance for 
example as discussed above], there would definitely be at least 3 different meta-data 
tags from the at least 9 which would not be addressed and therefore the neural network 
would be missing at least 3 different meta-data tag evidence). 

However, Tretter fails to specifically suggest that the network is a Bayesian 
network. 



Application/Control Number: Page 8 

10/712,657 
Art Unit: 2624 

Vivarelli discloses the network (Tretter's nodal neural network) is obtained by 
using a Bayesian neural network (see page 269, section B AYES IAN TRAINING OF 
NEURAL NETWORKS, paragraphs 1-2, Vivarelli's Bayesian neural network can replace 
Tretter's nodal neural network analysis for a final classification of a digital image). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Tretter's method using Vivarelli's teachings of 
making a final classification using a combination of both content based and meta data 
based classification through nodal neural analysis (see Tretter, paragraph [0009], lines 
1-11, paragraph [0010], lines 1-5) by replacing it with a Bayesian neural network in 
order to further improve the accuracy of the classification by applying the appropriate 
contribution weights to the two classifiers (Tretter's content based and meta-data based 
classification). 

Further, the inventor Jiebo Luo in the Declaration under 37 C.F.R. 1.132 filed 
10/09/2007, clearly states that a conventional Bayesian network is trained to be able to 
perform when data of input information is missing [see Declaration, page 1, Point 7]. 
Therefore the inventor clearly admits that the amended limitation is conventional in that 
a Bayesian network may perfonm with evidence missing. 

Re Claim 2: Tretter further discloses the metadata extracted in step (a) includes one or 
more of exposure time, aperture, shutter speed, brightness value, subject distance / 
focusing distance and flash fired / flash - no flash (see abstract, lines 6-1 0, paragraph 
[0008], lines 9-12). 
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Re Claim 3: Tretter further discloses the image content features in step (c) include one 
or more of color, texture and semantic features (see paragraph [0003], lines 7-8, 
paragraph [0008], lines 1-3, paragraph [0025], lines 6-11). 

As to claim 5, the claim is the corresponding computer readable medium for 
storing a program claim to claim 1 respectively. The discussions are addressed with 
regard to claim 1 . 

9. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Tretter, 
as modified by Vivarelli, and in further view of Schroder et al (US 7.020,330 B2, as 
applied In previous Office Action). The teachings of Tretter as modified by Vivarelli 
have been discussed above. 

However, Tretter as modified by Vivarelli falls to disclose or fairly suggest 
applying a customized image enhancement procedure to the digital image in response 
to the final estimate of image class of the digital image. 

Schroder discloses applying a customized image enhancement procedure / color 
correction process for assigned image class (3) to the digital image in response to the 
final estimate of image class (2) of the digital image (see Fig. 1 , abstract, lines 8-1 1 ). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to further modify Tretter's method, as modified by Vivarelli, 
by using Schroder's teachings by including the application of applying a customized 
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image enhancement procedure after image classification in order to apply the color 
correction process which is most likely for a particular image belonging to a particular 
selected image class (see Schroder, col. 3, lines 10-12). 

10. Claim 7 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over Tretter as 
modified by Vivarelli and Schroder, and further In view of Cooper ("A novel approach to 
color cast detection and removal in digital images" - SPIE - Jan 2000, vol 3963, pages 
167-177, as applied in previous Office Action). The teachings of Tretter as modified by 
Vivarelli and Schroder have been discussed above. 

However, Tretter as modified by Vivarelli and Schroder falls to disclose or fairly 
suggest the customized image enhancement procedure is color balancing and the 
customized image enhancement procedure includes retaining or boosting brilliant colors 
in images classified as sunset scenes and removing warm-colored cast from indoor 
images classified as tungsten-Illuminated scenes. 

Cooper discloses the customized Image enhancement procedure Is color 
balancing / color cast due to illuminant sources removal and the customized image 
enhancement procedure Includes retaining or boosting brilliant colors in images 
classified as sunset scenes / cast removal from outdoor sunset conditions and removing 
warm-colored cast from Indoor Images / cast removal from indoor natural images 
classified as tungsten-illuminated / unusual illuminant scenes (see title, abstract, line 1 , 
section 7 - RESULTS, paragraph 1, lines 5-6, paragraph 2, lines 1-3). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to further modify Tretter's method, as modified by Vivarelli 
and Schroder, using Cooper's teachings by including the color correcting color cast 
removal feature in order to correct the color cast which is encountered in indoor and 
outdoor images such as sunset images (see Cooper, section 7 - RESULTS, paragraph 

I, lines 5-6). 

Conclusion 

II. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Regazzoni discloses a Bayesian Network automatically deals 
with missing data. 

12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Bernard Krasnic whose telephone number is (571 ) 270- 
1357. The examiner can normally be reached on Mon-Thur 8:00am-4:00pm and every 
other Friday 8:00am-3:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on (571 ) 272-7429. The fax phone number for 
the organization where this application or proceeding Is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
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Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Bernard Krasnic 
January 22, 2008 



